This Markdown documents the process to analyze the data for Study 4 which looked at the difference between those who received and those who did not receive a warning at the beginning of the study and the effects of this warning after a 1 day delay.
Data was collected between August 7, 2018 and August 9, 2018 by using Amazon’s Mechanical Turk for ditribution and Qualtrics as a survey platform.
Set Up
Packages and Libraries
You must run this section before you can run any other chunks.
Data Import
Exclusions
You can exclude subjects who did not get the correct answer in the exercise by changing Exclude_Exercise_Check (line 99) to TRUE. The next time you run all the code, these participants will be excluded.
For this report, no participants are being excluded from analysis.
Methods
Participants
157 (53 women, Mage = 33.92, SDage = 9.97) completed both sessions of Study 4.
175 people completed session 1. 157 returned to complete session 2. 89.7142857142857% return rate.
All participants reported the United States as their location and had a previous task approval rate that was equal to or exceeded 85%.
74% reported having at least a Bachelor’s degree. The samples also exhibited a range of graph literacy (see Table 1).
Achieved power for main effects
##
## Paired t test power calculation
##
## n = 77
## d = 0.4
## sig.level = 0.05
## power = 0.9338806
## alternative = two.sided
##
## NOTE: n is number of *pairs*
Exercise Check
79 people did not complete exercise since they were in the NO WARNING condition. 2 other people did not complete the exercise at all.
16 did not get the manipulation check question right. Accuracy was 76.923%
For this report, no participants are being excluded from analysis. If you want to see results when participants who got the exercise wrong are excluded/included, you can go to the section called Exclusions (at the top of this file) and change Exclude_Exercise_Check <- TRUE
Results
The truncation effect
We first replicated our central effect of interest: the truncation effect.
We found that average ratings for truncated graphs was higher than ratings for control graphs: Mcontrol = 3.78, SD = 0.99; Mtruncated = 4.59, SD = 0.9.
##
## Cohen's d
##
## d estimate: -0.5338497 (medium)
## 95 percent confidence interval:
## lower upper
## -0.7598465 -0.3078529
##
## Paired t-test
##
## data: subject_mean_rating by graph_condition
## t = -15.972, df = 156, p-value < 0.00000000000000022
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.9168274 -0.7150198
## sample estimates:
## mean of the differences
## -0.8159236
This main effect was statistically significant: t(156) = 15.97, p < 1.204459e-34, 95% CI of difference = [0.72, 0.92], d = 0.53.
Most participants in both sessions showed an overall truncation effect: 88.54% of participants in Session 1 (139 of 157) and 85.35% of participants in Session 2 (139 of 157) .
The truncation effect across warning conditions
Does an explanatory warning reduce the size of the truncation effect, as seen in previous studies? To interrogate an observed interaction between graph condition and warning condition , we computed models for each warning condition separately. In these two models, graph type (0 = control, 1 = truncated) served as a binary fixed factor, and the judged difference between bars was the outcome variable. Participant and item were included as random effects.
No warning
In the no warning condition, we found a significant difference between control and truncated graphs (b = 1.08, SE = 0.03, t = 33.48, p < 4.824725e-226) such that truncated graphs, relative to control graphs, exaggerated the judged difference between bars. In the warning condition,
Warning condition
In the warning condition, we also found a significant difference between control and truncated graphs (b = 0.55, SE = 0.03, t = 16.49, p < 7.812787e-60) such that truncated graphs, relative to control graphs, exaggerated the judged difference between bars. Nevertheless, the magnitude of the judged difference between bars is larger in the no warning condition than in the warning condition.
Is an explanatory warning still protective 24 hours later?
To answer this, we computed linear models for Session 1 and Session. In these two models, graph type (0 = control, 1 = truncated) and warning condition (0 = no warning, 1 = warning) served as binary fixed factors, and the judged difference between bars was the outcome variable. Participant and item were modeled as random effects.
In session 1, we found a statistically significant main effect of graph type (b = 1.15, SE = 0.05, t = 24.85, p < 6.681194e-130) and a statistically significant interaction between graph type and warning condition (b = -0.59, SE = 0.07, t = -9, p < 3.061309e-19).
Critically, we saw a similar pattern of results in session 2: we found a statistically significant main effect of graph type (b = 1.01, SE = 0.05, t = 22.3, p < 6.061202e-106) and a statistically significant interaction between graph type and warning condition (b = -0.47, SE = 0.06, t = -7.29, p < 3.489533e-13).
Overall, these results (summarized in Figure 6) show that an explanatory warning results in a smaller truncation effect, and that this protective effect is still present 24 hours later.